API based web applications, distributed applications and client server applications may use one or more proxy nodes (including servers, virtual machines and linux containers) interposed between clients and servers for routing, load balancing and security at the API layer. Proxy nodes of the above type distribute incoming client requests or messages among multiple servers to ensure one or more of targeted routing, balanced server utilization, minimized server overloads, and high availability.
Proxies of router or load balancer type receive client requests or messages, select an appropriate server(s) for processing the requests or messages and transmit the requests or messages to the selected server(s).
FIG. 1 illustrates a network architecture 100 comprising a proxy 104 disposed as a network intermediate between clients 102 (i.e. clients 102a, 102b and 102c) and servers 106 (106a, 106b and 106c).
Based on information retrieved from a DNS server or other name server, requests or messages from client 102 for services from server backend 106 are directed to proxy 104. Proxy 104 transmits the received requests or messages to an appropriate server (106a to 106c) within server backend 106. Depending on the configuration of proxy 104, responses from servers 106a to 106c may first be received at proxy 104 and thereafter redirected to requesting client 102.
In implementing proxy functionality (for example, routing or load balancing), a proxy receives data packets addressed to a target service or server. The proxy transmits the data packets to an appropriate server based on predefined policies and techniques (e.g. routing or load balancing policies and techniques). In prior art proxies, such switching decisions typically rely on information associated with Layers 2 to 4 of the Open System Interconnection (“OSI”) model, or Transmission Control Protocol/Internet Protocol (“TCP/IP”).
FIG. 2 illustrates protocol layers of the OSI model 200 and the corresponding TCP/IP model. In the OSI model, each network device modularly implements the various OSI layers. Beginning with Layer 7 and progressing downward, each layer communicates with the layer immediately below and immediately above. As devices move to lower layers, the information is increasingly packaged for the specific hardware associated with the device/network—concluding in Layer 1, which is the physical communication channel. Under TCP/IP, Layers 1 to 2 are implemented as a subnet (or MAC) stack, Layer 3 as the Internet (or IP) stack, Layer 4 as the transport (or TCP/UDP) stack, and Layers 5 to 7 as the Application stack. Data generated by a first network device is processed down the protocol stacks, from Layer 7 to Layer 1, into a packet, with each stack adding a header to the packet. The packet is then transmitted over a physical channel to a second network device, which processes the packet up the stacks starting from Layer 1, and unwraps the respective headers after terminating them at their associated stacks. At Layer 7, the application data of the first device is retrieved for interaction with the application of the second device.
FIG. 3 illustrates various headers of an IP packet 300. Each IP packet consists of a data portion for carrying a data payload and a header portion for carrying overhead information. The header portion is partitioned into layer or protocol dependent headers. By way of example, a Layer 2 or MAC header includes a destination MAC address and a source MAC address that specify the destination and source hardware addresses of a node within a subnet. A Layer 3 or IP header includes a source IP address and a destination IP address that respectively specify the IP addresses of the source and destination nodes on the Internet. A Layer 4 or TCP header includes a source TCP port and a destination TCP port that respectively specify the port numbers used by the source node and the destination node.
The payload/data portion of the IP packet contains Layer 7 information which includes data generated by the application. This data may include HTTP headers—which is regarded as application data and is therefore located in the payload portion of the IP packet. The HTTP header may comprise a URL field for specifying a URL that the data packet is requesting, and may also include a cookie field for the application to communicate environmental information.
Implementing data analytics or security solutions at an individual server level has certain associated limitations, inasmuch that the pool of analyzed data is limited to data received at the individual server—which limits the reliability and accuracy of the results of such analysis.
Additionally, implementing first level security analysis at the server level is a less than optimal solution, for the reason that a server necessarily requires to receive a client request or message before it can determine whether such request or message comprises a security threat. The fact that a security threat necessarily has to be received at a server (thereby subjecting the server to the likelihood of damage) before the security threat can be identified or assessed, presents challenges of its own to server security.
Existing API security solutions also face challenges in terms of big data characteristics of velocity (increasing bandwidth and increasing processing powers of devices resulting in increasing demand for web services and APIs), volume (increasing number of API requests from increasing number of users having increasing number of devices), variety (increasing number of available web services and APIs) and veracity (identifying source and security or authentication related parameters of web service access data and/or API access data).
There is accordingly a need for efficient data analytics and security solutions that protect a server backend from security threats. Additionally, the security solutions need to be tailored for addressing the specific security requirements associated with big data capture, streaming and processing for protecting APIs at the edge of (i.e. at entry points of) cloud and data centers using aggregated information from various endpoints.